As it is known in the art, Internet Protocol Security (IPsec) is a security protocol that provides security services at the IP layer by enabling a system to select required security protocols, determine the algorithm(s) to use for services, and put in place any cryptographic keys required to provide the requested services. IPsec can be used to protect one or more paths between a pair of hosts, between a pair of secure gateways, or between a secure gateway and a host. The set of security services that IPsec can provide includes access control, connectionless integrity, data origin authentication, limited traffic flow confidentiality, and anti-replay checks.
In IPsec, two protocols are used to provide traffic security. These protocols utilize the Authentication Header (AH) and Encapsulating Security Payload (ESP). The IP Authentication Header is used to provide connectionless integrity and data origin authentication for IP datagrams, and to provide protection against replays. The ESP protocol may provide confidentiality (encryption) and limited traffic flow confidentiality. It also may provide connectionless integrity, data origin authentication, and an anti-replay service. The AH and ESP protocols may be applied alone or in combination with each other to provide a desired set of security services in IPv4 and IPv6.
IPsec offers, via AH or ESP protocol, a form of partial sequence integrity referred to as anti-replay integrity, which detects the arrival of duplicate IP datagrams (or packets) within a constrained window to help counter denial of service (DoS) attacks. A DoS attack is a type of attack on a network that is designed to bring the network to its knees by flooding it with useless traffic. Many DoS attacks, such as the Ping of Death and Teardrop attacks, exploit limitations in the TCP/IP protocols. The anti-replay mechanism seeks to overcome DoS attacks by assigning the sequence number of received packets, and dropping any packets having duplicate sequence numbers within a predefined window.
While the IPSec anti-replay mechanism is effective in many peer to peer environments, there are network architectures that generally cannot benefit from its application. One example of such a system is Provider Provisioned Virtual Private Networks (PPVPNs) such as those described in RFC 2547, incorporated herein by reference. Another such network is described in METHOD AN APPARATUS FOR PROVIDING A SCALABLE AND SECURE NETWORK WITHOUT POINT TO POINT ASSOCIATION, Ser. No. 10/661,959, filed on Sep. 12, 2003 by Fedyk et al, (hereinafter referred to as the Group Security Association (GSA) architecture).
Both the PPVPNs and the GSA architecture use group keys to efficiently secure data communications between edge devices. The GSA architecture incorporates the concepts of group key management protocol with a modification of security boundary positioning to provide a network that is both readily scalable and secure. Trusted ingress points and a trusted egress points in the network are identified. The trusted ingress point may be, for example, a gateway station, which attaches a source station (which is part of a local area network) to the internet, and it is assumed that the gateway station provides firewall protection to any communication behind the gateway. Similarly, the trusted egress station is any station that is coupled to a destination station, and wherein communication between the trusted egress station and destination station is protected. IPSec processing would also be performed at the egress station.
According to GSA architecture each station that is to be part of a private network registers with a key table. A group security association associated with the private network is forwarded to each trusted ingress and egress point that communicates with each member of the private network. When a member of the private network seeks to communicate with another member, it simply forwards the communication to the trusted ingress point with a Virtual Private Network (VPN) group address associated with the other member. The trusted ingress point uses the group security association associated with the private network to transform the communication and forwards the transformed communication through other intermediate stations in the network, until it reaches the trusted egress point. The trusted egress point uses the stored group security association corresponding to the Virtual Private Network (VPN) group address to decode the transformed communication and forwards the communication to the appropriate destination.
Thus, the GSA architecture provides a scalable means for securing communication between two end-points. However, the GSA architecture cannot easily support the use of sequence number for anti-replay handling, because multiple users share the same Security Association (SA), and the multiple users cannot easily synchronize the sequence number of the SA. It would be desirable to identify an anti-replay mechanism for use in the GSA architecture.